A Comparison of Surrogate Success Measures in Ongoing Representation Decision Support Systems: An Extension to Simulation Technology

New technology and software development have provided corporate managers with a wide variety of decision making aids. One such tool, classified as a representational decision support system, is discrete event computer simulation. In order to assess the organizational impact of discrete event computer simulation, an instrument capable of measuring success is required. The importance of such assessment cannot be overemphasized. While empirical measurement of various information system inputs or independent variables such as information system budget expenditures or user participation is relatively straightforward, the development of corresponding output or dependent variables has been difficult. In an attempt to overcome these difficulties, researchers have suggested a variety of measurable surrogates. Work in this area has paved the way for the development of instruments used to assess success. This paper focuses on external validity aspects of two popular information system instruments, the Davis measure of User Acceptance of Information Technology and the Doll and Torkzadeh measure of End-User Computing Satisfaction (EUCS). These instruments were designed for general purpose use and tested across a variety of settings, times, and persons. To ensure this generalizability extended to a very specific form of information technology, these instruments were administered to discrete event computer simulation users and tested for psychometric stability. This study provides additional evidence that the Doll and Torkzadeh measure of End-User Computing Satisfaction retained its psychometric properties when applied to users of discrete event computer simulation and therefore provides a reasonable surrogate measure for success in the implementation of this technology. An initial assessment of the Davis measures of User Acceptance of Information Technology (Perceived Ease-of-Use, Perceived Usefulness) returned poorer scores on the fit indexes, but the evidence did indicate the expected factor structure was supported to some extent. The managerial implications of these findings are discussed.